feat: migrate AKernel deployment to Python YuanRong CLI - #19
feat: migrate AKernel deployment to Python YuanRong CLI#19mhsong1998-dot wants to merge 8 commits into
Conversation
b7de90a to
7e5c006
Compare
7e5c006 to
6a7e84d
Compare
|
One remaining process issue before merge: the current commit body contains only the Also, GitHub currently reports no checks for this branch. The validation documented in the PR is extensive, but there is no automated status attached to the commit. Please ensure the expected CI checks are configured and passing before merge, or explicitly confirm that this repository does not currently provide PR checks. |
3d195f7 to
317abec
Compare
ae77634 to
1e30b39
Compare
|
Process items are addressed in current HEAD 1e30b39: the branch contains one lowercase Conventional Commit with a prose body, and its author, committer, and Signed-off-by trailer all match mhsong1998-dot. GitHub created CI run 32334113441 for this head, but currently marks it action_required, so a maintainer must approve the fork workflow before the checks can execute. The previously approved head passed all eight CI jobs. |
bfa5b8b to
4d55ac6
Compare
4d55ac6 to
1852878
Compare
| @@ -0,0 +1,30 @@ | |||
| # Runtime dependency lock for openyuanrong-core 0.10.1 on Python 3.12. | |||
There was a problem hiding this comment.
Why do we need this file?
There was a problem hiding this comment.
Kept. This file was added in response to the earlier reproducibility review (discussion_r3756876467): the core wheel metadata uses lower-bounded transitive dependencies, so pinning the complete Python 3.12 dependency closure prevents the same AKernel commit from resolving different versions over time. It is updated together with OPEN_YR_VERSION.
| fi | ||
| fi | ||
|
|
||
| case "${role}" in |
There was a problem hiding this comment.
Seems to be a duplicate of line 10.
There was a problem hiding this comment.
Updated in 7485d18. The entrypoint now consumes the role argument once, validates the resulting role once, and then exports AKERNEL_ROLE; the duplicate role-selection case is gone.
| ;; | ||
| node) | ||
| export DEPLOY_PATH="${DEPLOY_PATH:-/home/yuanrong}" | ||
| export YR_LOG_PATH="${YR_LOG_PATH:-${DEPLOY_PATH}/logs}" |
There was a problem hiding this comment.
These two environments should not be set in this file.
Please move them to the yuanrong setup bash.
There was a problem hiding this comment.
Updated in 7485d18. DEPLOY_PATH and YR_LOG_PATH are no longer set by akernel-entrypoint; the node/standalone YuanRong bootstrap remains the single place that supplies these defaults.
| ;; | ||
| standalone) | ||
| export DEPLOY_PATH="${DEPLOY_PATH:-/home/yuanrong}" | ||
| export YR_LOG_PATH="${YR_LOG_PATH:-${DEPLOY_PATH}/logs}" |
There was a problem hiding this comment.
Seems it's a duplicate set in the Yuanrong startup script.
There was a problem hiding this comment.
Updated in 7485d18 together with the node branch. The duplicate standalone assignments were removed, and the YuanRong bootstrap owns these defaults.
| sleep 1 | ||
| done | ||
|
|
||
| echo "timed out after 60s waiting for sandbox0 to have an IPv4 address" >&2 |
There was a problem hiding this comment.
Not an elegant way. Any method to resolve this dependency? For example, can we set the yuanrong service to start after the sandboxd service? And the IP can be obtained from the ‘ip_range’ config.
There was a problem hiding this comment.
Updated in 7485d18. sandboxd.service now has a bounded ExecStartPost readiness check for the live sandbox0 IPv4, and yuanrong.service explicitly Requires/starts After sandboxd. The YuanRong bootstrap therefore reads the address once instead of polling. The address is intentionally not derived from ip_range because the earlier review in discussion_r3734794038 required the actually assigned interface address as the source of truth.
| ARG FIRECRACKER_RELEASE | ||
| ARG FIRECRACKER_AMD64_SHA256 | ||
| ARG FIRECRACKER_AMD64_URL | ||
| ARG KATA_LICENSE_URL=https://raw.githubusercontent.com/kata-containers/kata-containers/${KATA_RELEASE}/LICENSE |
There was a problem hiding this comment.
Is it required for our deployment change? Why change the kata things?
There was a problem hiding this comment.
Updated in 7485d18. The Kata license URL parameter and local download-timeout adjustment were unrelated build-network accommodations, so this block is restored to the current main-branch implementation.
| echo "${wheel_sha} ${wheel}" | sha256sum -c -; \ | ||
| python3 -m pip install \ | ||
| --break-system-packages \ | ||
| python3 -m venv /opt/openyuanrong; \ |
There was a problem hiding this comment.
Why we need to use venv?
There was a problem hiding this comment.
The venv is retained because the Python CLI wheel now installs its complete runtime dependency set. It keeps those dependencies isolated from Ubuntu system Python and avoids mutating the externally managed system environment with --break-system-packages.
| sed -i 's/"env_vars": comp.env_vars,/"env_vars": {},/' "${launcher_py}"; \ | ||
| grep -Fq '"env_vars": {},' "${launcher_py}"; \ | ||
| ! grep -Fq '"env_vars": comp.env_vars,' "${launcher_py}"; \ | ||
| test -x /opt/openyuanrong/bin/yr; \ |
There was a problem hiding this comment.
Those changes for what needs?
There was a problem hiding this comment.
Retained as a security hardening required by the earlier review in discussion_r3756876315. The 0.10.1 wheel still logs the full component environment and persists component env_vars in session JSON. The exact grep checks make the build fail if an upstream source change causes either redaction patch to stop applying, rather than silently exposing values such as LITEBUS_DATA_KEY.
| default) | ||
| - Print the Traefik container IP to use as `AKERNEL_SERVER_ADDRESS` | ||
|
|
||
| The openYuanRong bootstrap waits for `sandbox0` and uses the IPv4 address |
There was a problem hiding this comment.
Removed in 7485d18. This implementation detail remains documented in the repository maintenance guidance and does not need to be part of the standalone user quick-start.
1852878 to
7485d18
Compare
7485d18 to
72ce2b6
Compare
Replace the legacy Go launcher with the Python CLI for master, frontend, node, and standalone while preserving fixed deployment and log paths. Keep main's 0.10.1rc4 artifacts, legacy actor SDK pin, checkpoint settings, and RRT control socket environment when rebasing the migration. Wait for the live sandbox bridge before starting YuanRong and handle an interface that has not appeared yet. Preserve the 10.88.0.1/16 network default across Helm and Terraform, and cover startup readiness and role log paths in deployment CI. Signed-off-by: mhsong1998-dot <258010372+mhsong1998-dot@users.noreply.github.com>
Sandboxd stays in the foreground and does not create the configured PID file. With ExecStartPost readiness enabled, systemd waits for that missing file, times out startup, and prevents YuanRong from starting. Use an explicit simple service so systemd tracks the ExecStart process. Verified a fresh standalone startup and the basic usage, command stdin, named sandbox, and PTY examples on native amd64 with the Python CLI. Signed-off-by: mhsong1998-dot <258010372+mhsong1998-dot@users.noreply.github.com>
Signed-off-by: mhsong2 <songminhui2@huawei.com>
Signed-off-by: mhsong2 <songminhui2@huawei.com>
Signed-off-by: mhsong2 <songminhui2@huawei.com>
Signed-off-by: mhsong2 <songminhui2@huawei.com>
Signed-off-by: mhsong2 <songminhui2@huawei.com>
Signed-off-by: mhsong2 <songminhui2@huawei.com>
b3dc80d to
7130e71
Compare
3 participants
Summary
Migrate master, frontend, node, and standalone startup from the legacy Go launcher to the Python openYuanRong CLI. Preserve fixed deployment paths, role-specific logs, external sandboxd resource collection, distinct Pod/bridge addresses, DataSystem health and timeout settings, checkpoint storage, and
YR_RRT_CONTROL_SOCKET_PATH.Rebased onto
mainat5be301df1333064571b8e4acbee21bc4535d05e6. The conflict resolution keeps the currentYR_IMAGE_PROCESS_CONFIGstartup contract and inherited-image-entrypoint support from main. The sandboxd gitlink points to inclusionAI/sandboxd#48 atbf42f63b2916afdcb6414374469df713c8209381.Startup and storage behavior
/opt/openyuanrong/bin/python /opt/openyuanrong/bin/yr ... start; master/frontend add--masterand all roles keep blocking foreground supervision.Type=simple.ExecStartPost=/usr/local/bin/sandboxd-network-readywaits untilsandbox0has an IPv4 address or reaches its bounded timeout, including the case where the interface does not exist yet.Validation
Local
make SHELL=/bin/bash deploy-script-checkpython3 -m unittest discover -s builder/tests -v(6 passed)helm lint deploy/akernel/charts/corego test ./pkg/imagemanager/distillfsin the sandboxd submoduleComponentized CCE deployment
Validated head
7130e71b836455a031f64ead99e9f76768b01a5bincn-north-4/akernel-cce, namespaceakernel, Helm releaseakernel-corerevision 52.swr.cn-north-4.myhuaweicloud.com/openyuanrong/cluster-all-in-one:pr19-7130e71sha256:d3f90dda928de51131b386acf6520b193817f3da6396308430b2cf012c36065esandboxd.serviceandyuanrong.serviceare active on all four nodes; every node has a livesandbox0IPv4 address.yrCLI.Python examples
Ran all 14 files under
sdk/python/examplesusing SDK client imageswr.cn-north-4.myhuaweicloud.com/openyuanrong/akernel-sdk-client:pr19-7130e71(sha256:3b636bbe50dc70dd8109f0990fa68e32eeb9b77c19037adaa83efaaa71db08ad).basic_usage,command_stdin,failover_reload,named_sandbox,network_policy,pty,reverse_tunnel,s3_rootfs_and_mounts, andstorage_sandbox.custom_image,dockerfile_launch, andport_forwardingtimed out while resolving their default public Docker Hub images. The same three examples passed without code changes when only their image addresses were switched to mirrored SWR images; all ninedockerfile_launchsections passed.gpu_sandboxreached scheduling and returnedERR_RESOURCE_NOT_ENOUGHbecause this cluster has no matching GPU card.sandbox_runtimepassed the default and runsc paths, then reported that Kata is unsupported on these nodes.basic_usageattempt immediately after rollout hit the old frontend Pod IP once. A separate stable-state rerun passed the full create, command, file, background-process, and cleanup flow.All temporary Jobs, Services, ConfigMaps, Secrets, and the S3 fixture were deleted after validation; all four node sandbox lists are empty.